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Abstract 

This study investigated the use of Pathfinder Associative Networks in assessing knowledge 
structures for people who learn information via computer-based tutorials. Graphical 
representations for a set of concepts were produced for participants that had completed a 
computer-based tutorial related to the concepts (novices) and for those that had no tutorial 
(naives). These graphs were then compared to graphical representations generated by experts. 
Comparing graphs revealed that the naives and novices were the most similar groups. However, 
the differences that occurred between these two groups were indicative of the fact that novices 
were becoming more similar to experts. 
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Assessment of Training Using Pathfinder Associative Networks 

Currently in the Health Care field, professionals in the area of training use computer- 
based tutorials as one way to present new information to fieldworkers such as mental health 
technicians and community health educators. Following the tutorial, the fieldworker will 
typically complete a test, usually multiple-choice, to demonstrate that he or she has learned the 
material. However, it is unclear whether the fieldworkers are actually learning from these 
tutorials. It is possible that the fieldworker completing the tutorial passes the multiple-choice test 
simply because of the recency of the presentation of the material. In this case the tutorials would 
be testing very recent memories but not the degree to which the fieldworkers integrated the 
presented material. In addition, this form of assessment may not be adequate for those who lack 
test-taking skills or those who get anxious while taking tests. Therefore, professionals in the area 
of training for these fieldworkers need a better way to assess whether the material is truly 
learned. 

An alternate method of assessing whether the material is truly learned is through 
assessment of how the knowledge in the tutorial is organized. It is already known that 
knowledge is organized as it is obtained. In fact, since the time of the ancient Greeks, it has been 
known that learning requires organized efforts (Walker, 1996). If we can use a quantifiable 
method for determining how a person organizes a particular set of concepts, we may be able to 
use that method for assessing how material in tutorials is learned. The Pathfinder Network 
Scaling Algorithm is just such a method. 

Pathfinder networks are graphical representations based on similarity ratings of a set 
concepts in a knowledge space. Pathfinder uses the similarity ratings to create a data set called a 
proximity matrix. The matrix is then translated into a graph denoting similarities between 
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concepts as distances on the graph. The graphical representation is in the form of a model 
consisting of nodes and links (see Schvaneveldt, Durso, Goldsmith, Breen, Cooke, Tucker, & 
DeMaio, 1985 for a more detailed description). Using these graphs, we can compare knowledge 
structures of similar problems for people under different conditions. 

Previous research indicates that Pathfinder can successfully be applied to issues in 
education. Gomez, Hadfield, and Housner (1996) used Pathfinder to examine knowledge 
obtained during a teaching methodology course in elementary mathematics. In order to further 
investigate the understanding of novice-expert differences in elementary math education, as well 
as methods for enhancing and developing conceptual and pedagogical content knowledge, the 
researchers assessed and compared the knowledge structures of the course instructor, teacher 
educators, and prospective teachers. They found that the course instructor shared a high degree 
of structural knowledge with other experienced teacher educators. In addition, the prospective 
teacher networks were similar to those of the course instructor and experienced teacher 
educators, suggesting that prospective teachers are gaining a more general knowledge of 
pedagogy. 

In another study, Pathfinder was shown to be a more sensitive predictor of teaching 
performance than standard exams. Goldsmith, Johnson, and Acton (1991) used Pathfinder to 
assess students’ structural knowledge of classroom learning. Students enrolled in a college 
course on psychological research techniques were examined to see if students with similar 
structures perform at a similar level on exams. Using Pathfinder they found that good students 
had similar knowledge structures and that poorer students have less similar knowledge 
structures. These findings are consistent with the notion that novices are unable to organize 
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information in meaningful ways, whereas the experts are able to organize information in terms of 
abstract relational schemata that are unavailable to novices. 

Because Pathfinder was successfully used in education to assess knowledge structures, 
we decided to use Pathfinder to exa min e the knowledge structures of those who completed a 
tutorial on a health care related topic. The current study was designed to answer two questions: 
Are the tutorials actually good learning devices and can Pathfinder be used to assess this 
effectively? It was hypothesized that one could expect to find a positive relationship between the 
knowledge structures of those who have had the tutorial and that of experts in the field. 

Method 

Participants 

Thirty undergraduate students enrolled in general psychology at a Southern university 
volunteered to participate. These students were given extra credit for their participation. Nine 
clinical and counseling experts from the psychology and counseling department and the 
university’s counseling center also volunteered to participate. 

Materials 

Tutorial. The computer-based tutorial that students in the training condition received is 
part of a multi-modal self-instruction program series on mental illnesses and behavior disorders 
for people who have mental retardation (Poindexter, 1997). An individual tutorial on the topic of 
schizophrenia and other psychotic disorders was used for this study. The program provides the 
material in small blocks. After each block the participants answer questions about what they just 
read. If they answer correctly they move to the next block. If they answer incorrectly the 
program takes them back to the block to review the material again. 
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Rating task. All participants, regardless of condition, rated concepts from the tutorial for 
similarity using Pathfinder. Pathfinder pairs concepts together and prompts the participant to 
decide how similar the two concepts are on a scale of one (not similar) to nine (very similar). 

The participants rated all possible pairwise comparisons of concepts. 

Distractor task. The control group did not receive the tutorial. Instead, they completed a 
distractor task consisting of five puzzle mazes. The students’ task was to trace a continuous line 
from the top left corner to the bottom right corner of the same maze without crossing any lines in 
the maze. 

Procedure 

Before the experiment began, concepts for the rating' task were selected by asking five 
people to complete the tutorial and record what they believed were key concepts. Then they 
were asked to rank those concepts in order of importance. The 13 concepts that all people agreed 
upon were selected to be used for the rating task (see Appendix). 

For the experiment, student participants were randomly assigned to one of two 
conditions. In the Experimental condition (novice), participants first navigated through the 
tutorial, then when finished rated the concepts for similarity. In the Control condition (naive), 
participants first completed the maze task, and then completed the rating task. Faculty and 
professional staff members (expert) completed the rating task only. 

Results and Discussion 

Graphical Representation 

The similarity scores for each individual were averaged among all members per condition 
before applying the Pathfinder algorithm to the ratings. Figures 1 through 3 show the Pathfinder 
graphs for each condition. 
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Central Tendency 

Pathfinder graphs can be assessed in terms identifying focal nodes. Focal nodes are 
nodes of central tendency (see Dayton, Durso, & Shepard, 1990 for a detailed explanation of 
focal nodes). There are three ways to determine central tendency. First, we can identify central 
tendency in terms of the highest degree node, defined as the node that has the greatest number of 
links to it. Second, we can describe central tendency by the median node, which is the node with 
the smallest average distance to all other nodes. Third, we measure central tendency with the 
center node defined as the smallest maximum distance to any other node. Each type of central 
tendency and the corresponding nodes for all three nodes are represented in Table 1. 

Table 1 suggests that naive participants believe that the central concept for this set of 
concepts is medication. Expert graphs, on the other hand, tend towards psychosis as a central 
concept. Novice central tendency is different than both the naive participants and the experts. In 
contrast to naive participants, novices have viewed the tutorial and realize that medication is not 
central to the set of concepts. However, novices still have not formed a representation similar to 
the experts. This is evident in the fact that depression appears to be more central than psychosis. 
Network Similarity 

Comparison among conditions can also be made via similarity scores. Similarity scores 
are calculated by dividing the number of shared links between two graphs by the total number of 
links for those two graphs. In essence, this number represents the amount of overlap for the 
graphs in question. The network similarity score for the naive condition compared to novice 
condition was .444. The network similarity score for the naive condition compared to the expert 
condition was .136. The network similarity score for the novice condition compared to the 
expert was .19. 
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These similarity scores suggest that novices and naives are very similar in their structure 
of knowledge, yet novices are more similar to experts than naives. This type of finding should 
not be surprising. Surely, after one exposure to a set of information, we would not expect 
novices to be equal to experts in how the set of knowledge is organized. What is encouraging is 
that the novice is slightly closer to the expert score than the naive score when compared to the 
expert score, suggesting that some learning did occur. 

General Discussion 

In response to the earlier question, are individuals learning, the response is “yes”. 
However, it may be difficult to determine from this study that fieldworkers would benefit from 
this tutorial because fieldworkers may have some previous knowledge about the topic in the 
tutorial, unlike the students enrolled in general psychology. 

In terms of our other question, the usefulness of Pathfinder as a tool to assess learning, 
we believe we have demonstrated Pathfinder to be an effective method. Our findings suggest 
that Pathfinder detects the change in knowledge structures that even a small amount of training 
will produce. 

One possible use of Pathfinder that we have not addressed in this study involves using 
Pathfinder to track changes in knowledge organization as knowledge is accrued. We know from 
the current study that some learning has occurred after the first tutorial, but we do not know the 
course of development of expertise over a longer period of time in terms of how knowledge is 
organized. One way to test this would be to ask psychology majors to complete the ratings task 
on psychology concepts as freshman and then test them periodically until they reach their final 
degree level. We should be able to see changes in knowledge structures as expertise is 
developed, giving us insight to the process of becoming an expert. 
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Another question that we have not addressed in this study is whether the knowledge 
structure created by the tutorial remains the same over time. For example, we know that if we 
train someone on a body of knowledge and then test them using the multiple-choice format 
immediately after study, the test scores are high. If we test again later, say six months, the test 
scores are significantly lower due to forgetting. However, it is possible that when facts cannot be 
recalled, the knowledge is still represented in the same way. If so, then Pathfinder graphs 
generated immediately after studying a body of knowledge should be comparatively the same as 
graphs generated six months later by the same person. If not, then how we think about forgetting 
(loss of information) could change dramatically. Forgetting would have to be redefined as loss 
of information and loss of organizational structure. The use of Pathfinder Networks as an 
assessment tool will allow us to answer these and other questions. 
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Table 1. 

Central Tendency Nodes bv Level of Expertise 







Central Tendency Nodes 




Level of Expertise 


Highest Degree 


Center 


Median 


Naive 


Medication 


Medication 


Medication 


Novice 


Depression 

Medication 


Depression 

Medication 

Psychosis 


Depression 


Expert 


Psychosis 


Catatonic Behavior 
Affective Flattening 


Psychosis 
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Appendix 



Psychosis 

Delusions 

Hallucinations 

Mental Retardation 

Disorganization 

Catatonic Behaviors 

Depression 

Diagnostic Overshadowing 

Negative Symptoms 

Fantasies 

Medication 

Affective Flattening 

Positive Symptoms 
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